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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BEFORE THE BOARD OF PATENT APPEALS AND INTERFERENCES 



On Appeal to the Board of 
Appeals and Interferences 



Applicant 
Serial No. 
Filed 
Title 



Eleftheriadis et al. 
09/367,433 
January 13, 2001 



Group Art Unit : 2614 
Examiner : Jean W. Desir 



OBJECT-BASED AUDIO- VISUAL TERMINAL 
AND BIT STREAM STRUCTURE 



BRIEF ON APPEAL 



This brief on appeal is filed in response to a Final Office Action issued by the 
U.S. Patent and Trademark Office (the "PTO") on February 23, 2006. 

On May 25, 2006, Appellants filed a Notice of Appeal in the above-identified 
patent application from the final rejection of claims 1-10. In accordance with 37 C.F.R. § 41.37, 
this Appeal Brief is submitted in support of the Appeal of the final rejection. The fee for this 
Appeal as set forth in 37 C.F.R. § 41.20(b)(2) is provided herewith. For the reasons set forth 
below, the final rejection of pending claims 1-10 should be reversed. 

I. REAL PARTY IN INTEREST 

The real party in interest is The Trustees of Columbia University in the City of 
New York, by way of assignment from the named inventors, recorded on January 13, 2000, at 
Reel 010525, Frame 0615. 
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II. RELATED APPEALS AND INTERFERENCES 

Appellants and the Appellants' legal representatives are unaware of any appeals 
or interferences related to the present application which will directly affect or be directly affected 
by or have a bearing on the Board's decision in the pending appeal. 

III. STATUS OF CLAIMS 

In the February 23, 2006 Final Office Action, claims 1-10 were rejected under 
35 U.S.C. § 102(e) as allegedly being anticipated by U.S. Patent No. 5,826,102 to Escobar et al. 
(hereinafter "Escobar"). Appellants respectfully traverse the rejections of record. 

A copy of all of the pending claims is attached hereto in the Claims Appendix at 

page A-l. 

IV. STATUS OF AMENDMENTS 

Subsequent to the issuance of the Final Official Action dated February 23, 2006, 
no further amendments to the claims have been filed by Appellants. 

V. SUMMARY OF CLAIMED SUBJECT MATTER 

The present application relates generally to the representation, transmission, 
processing and display of video and audiovisual information. {See, e.g., Specification, page 1, 
lines 1-5; p. 10, lines 10-16; p. 15, lines 5-13). 

The claimed subject matter described in the above-identified application is 
directed to methods and systems for displaying object-based audiovisual/video data. More 
specifically, independent claim 1 and its corresponding dependent claims 2-7 are directed to 
methods for displaying object-based audiovisual/video data, comprising: 

(a) streaming in a data bit stream, over time, a plurality of 
audiovisual/video objects and composition information for the objects, 
to a receiver; 

(b) at the receiver, storing in a cache memory at least one of the objects; 
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(c) processing the composition information received in the data bitstream 
to compose scenes from said objects including the one of the objects 
stored in the cache memory; and 

(d) displaying the composed scenes. 
(Claim 1). 

Exemplary support for these claim limitations can be found throughout the specification and 
figures, including, e.g., as follows: 

(a) streaming in a data bit stream, over time, a plurality of 
audiovisual/video objects and composition information for the objects, 
to a receiver; [exemplary support for these limitations may be found at, 
e.g., specification, p. 4, lines 5-1 1 ("A composition packet contains an 
object's ID, time stamps and the 'composition parameters' for rendering 
the object. An object data packet contains an object ID, an expiration time 
stamp in case of persistent objects, and object data."); Figs. 3-5] 

(b) at the receiver, storing in a cache memory at least one of the objects; 
[exemplary support for these limitations may be found at, e.g., 
specification, p. 1 1, lines 21-24 ("The terminal further includes input 
buffers 3. AV object decoders 4, buffers 4' for decoded data, a composer 
5, a display 6, and an object cache 7."); p. 12, lines 10-15 ("In the object 
cache 7, objects are stored for use beyond their initial presentation time. 
Such objects remain in the cache even if the associated node is deleted 
from the scene graph, but are removed only upon the expiration of an 
associated time interval called the expiration time stamp.")]; 

(c) processing the composition information received in the data bitstream 
to compose scenes from said objects including the one of the objects 
stored in the cache memory; and [exemplary support for these limitations 
may be found at, e.g., specification, p. 13, lines 5 - p. 14, line 31]; 

(d) displaying the composed scenes, [exemplary support for these 
limitations may be found at, e.g., specification, p. 13, lines 5-20] 

Similar limitations, but in apparatus form, are provided in independent claim 8, including, e.g.,: 

(a) a controller circuit for controlling acquisition over time of streaming 
data including a plurality of audio visual/video objects and composition 
information for the objects; 

(b) a cache memory for storing at least one of the objects; 
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(c) a composer circuit, coupled to the cache memory, for processing the 
composition information acquired in the streaming data to compose scenes 
from said video objects including the one of the objects stored in the cache 
memory; and 

(d) a display for the composed scenes. 
(Claim 8). 

Also, as recited in claim 9: 

(a) controlling acquisition over time of streaming data including a plurality 
of audio-visual/video objects and composition information for the objects; 

(b) storing in a cache memory at least one of the objects; 

(c) processing the composition information acquired in the streaming data 
to compose scenes from said video objects including the one of the objects 
stored in the cache memory; and 

(d) displaying the composed scenes. 
(Claim 9). 

Finally, as recited in claim 10: 

(a) means for controlling acquisition over time of streaming data including 
a plurality of audio-visual/video objects and composition information for 
the objects; 

(b) means for storing in a cache memory at least one of the objects; 

(c) means, coupled to the cache memory, for processing the composition 
information acquired in the streaming data to compose scenes from said 
video objects including the one of the objects stored in the cache memory; 
and 

(d) means for displaying the scenes. 
(Claim 10). 

Generally, the application relates to challenges in the art of video 
encoding/decoding. In some embodiments of the claimed subject matter, information to be 
processed at an object-based video or audio-visual (AV) terminal includes an object-oriented 
bitstream comprising objects, composition information, and scene demarcation information. The 
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bitstream structure allows on-line editing, e.g., cut and paste, insertion/deletion, grouping, and 
special effects. (Specification, p. 1, line 27 - p. 2, line 3). 

VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Claims 1-10 were rejected under 35 U.S.C. § 102(e) as allegedly being anticipated 
by U.S. Patent No. 5,826,102 to Escobar et al. (hereinafter "Escobar"). Appellants respectfully 
request review of all rejections of record. 

VII. ARGUMENT 

A. The Rejections Under 35 U.S.C. § 102(e) in view 
of Escobar Should Be Reversed 

Pending claims 1-10 have been finally rejected under 35 U.S.C. § 102(e) as 
allegedly anticipated by U.S. Patent No. 5,826,102 to Escobar et al. ("Escobar"). Appellants 
respectfully traverse these rejections. 

To establish an anticipation rejection, the cited reference must disclose every 
element of the claimed invention. 35 U.S.C. § 102(e) states, in pertinent part, that "[a] person 
shall be entitled to a patent unless the invention was described in (1) an application for patent . . . 
by another filed in the United States before the invention by the applicant for patent or (2) a 
patent granted on an application for patent by another filed in the United States before the 
invention by the applicant for patent." A patent claim is thus anticipated under Section 102(e) if, 
among other things, "identity of invention" is shown. Minnesota Mining and Manufacturing Co. 
v. Johnson & Johnson Orthopedics, Inc., 976 F.2d 1559, 1565, 24 U.S.P.Q.2d 1321 (Fed. Cir. 
1985). In finding identity of invention, the PTO "must show that each element of the claim in 
issue is found ... in a single prior art reference." Id. The Federal Circuit has held that, "[a] 
claim is anticipated only if each and every element as set forth in the claim is found, either 
expressly or inherently described, in a single prior art reference." Verdegaal Bros. v. Union Oil 
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Co. of California, 814 F.2d 628, 631, 2 U.S.P.Q.2d 1051 (Fed. Cir. 1987). Moreover, u [a] prior 

art publication cannot be modified by the knowledge of those skilled in the art for purposes of 

anticipation." In re Saunders, 444 F.2d 599, 602-03, 170 U.S.P.Q. 213 (C.C.P.A. 1971). 

Escobar does not disclose or suggest a technique for displaying object-based 

audiovisual/video data, including, among other things, "streaming in a data bit stream, over time, 

a plurality of audiovisual/video objects and composition information for the objects, to a 

receiver" as recited in claim 1, or "controlling acquisition over time of streaming data including a 

plurality of audio-visual/video objects and composition information for the objects," as recited in 

claims 8-10. Escobar further fails to disclose or suggest any techniques for "processing the 

composition information received in the data bitstream to compose scenes from the received 

objects" as recited in claim 1, or "processing the composition information acquired in the 

streaming data to compose scenes from said video objects including the one of the objects stored 

in the cache memory," as recited in claims 8-10. As discussed more fully herein below, because 

Escobar fails to disclose or suggest at least these claimed features, Appellants' respectfully 

submit that Escobar cannot anticipate the claimed invention and that, accordingly, all rejections 

of record should be reversed. 

1. Claims 1-7 Are Not Anticipated Because Escobar 
Does Not Disclose "streaming in data bit stream, 
over time, a plurality of audiovisual/video objects and 
composition information for the objects, to a receiver 9 ' 

Appellants' claim 1 recites "streaming in a data bit stream, over time, a plurality 
of audiovisual/video objects and composition information for the objects, to a receiver." 

As disclosed in the specification, e.g., at p. 6, lines 10-14, the composition 
information may include composition parameters such as displacement from the upper left corner 
of the presentation frame, rotation angles, zooming factors. (FIGS. 2a-2c, and page 6 lines 10- 
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14), and thus is directed to where to position an object in a scene. Claim 1 thereby relates not 
merely to the use of encoded object-based structures in audio- visual transmission and scene 
display, but also to the use of the bitstream structure to convey, over time, composition 
information. No such arrangement is disclosed in or suggested by Escobar, 
a. Composition Information 

In contrast, Escobar merely describes using a computer to make a sequential time 
display of multimedia assets/objects on a time line. "Playback or execution of the [assets/] 
objects represented on the timeline occurs in a time sequence indicated by their position on the 
timeline." (See, e.g., col. 4 lines 5-20). Escobar does not disclose or suggest the claimed 
"composition information" because Escobar is not concerned with organizing the contents of a 
scene and does not show or describe composing a scene (e.g., positioning objects in a scene). 
Escobar only describes the delivery and presentation of multimedia applications using timelines 
to integrate various multimedia and program assets. (See, e.g., col. 7 lines 35-48 ("video assets 
from a video recorder, a variety of sounds recorded by an audio engineer, the film output from 
movie cameras with or without related audio tracks, and perhaps different films from different 
camera locations of the same scene shot by different cameras")). 

Accordingly, Escobar fails to disclose or suggest composition information as 
recited in Appellants' claim 1, and therefore cannot and does not disclose "streaming in a data bit 
stream, over time, a plurality of audiovisual/video objects and composition information for the 
objects, to a receiver," as recited in claim 1. For at least these reasons, Escobar cannot anticipate 
claim 1, and the rejections of record should be reversed. 
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b. Audiovisual /Video Objects 

Additionally, Escobar fails to disclose or suggest the feature of "streaming in a 
data bit stream, over time, a plurality of audiovisual/video objects ," as recited in claim 1. As 
described in Appellants' specification, e.g., at pg. 1, lines 10-13, audiovisual/video objects "have 
visual relevancy, representing familiar physical objects, e.g., a ball, a table, a person, a tune or a 
spoken phrase." No such audiovisual/video objects are disclosed in or suggested by Escobar. 

Escobar's multimedia assets/objects are not encoded audiovisual/video objects 
streamed in a data bitstream. Unlike the subject matter of claim 1, Escobar is not concerned with 
the processing of object-encoded audiovisual compression data (i.e., bitstream objects) of a scene 
for enhanced display at an audiovisual terminal. The "objects" or "assets" of Escobar (including 
film tracks, audio tracks, and the like) are not objects as recited in claim 1, as they could not be 
placed within a scene in the manner recited in claim 1 . For at least this additional reason, 
because Escobar fails to disclose encoded objects in accordance with Appellants' claimed subject 
matter, the rejections of record should be reversed. 

c. Streaming 

Escobar does not disclose or suggest " streaming in a data bitstream" both 
audiovisual/video objects and composition information as recited in claim 1. Rather, Escobar 
describes using edit decision lists (EDL) and interactive decision lists (IDL), which capture the 
editing decisions made by a user of an authoring tool to control timing of playback or execution 
of the multimedia assets/objects. (See, e.g., col. 4 lines 24- 36). Escobar describes storing 
IDL/EDL as an application as an ASCII file. (See, e.g., col. 10 lines 52-55). As such, Escobar 
does not disclose streaming of composition information or objects (or even IDL/EDLs) to a 
client/receiver. For at least this additional reason, because Escobar fails to disclose or suggest at 
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least the claimed "streaming in data bit stream, over time, a plurality of audiovisual/video 
objects," Escobar cannot anticipate claim 1. 

Appellants respectfully submit that the Office Action (page 2) mistakenly cites 
Escobar col. 4, lines 33-39, col. 6, lines 37-40 and col. 12, lines 46-47, and Escobar col. 6, 
lines 41-45, col. 4, lines 16-18 and col. 21, lines 26-31 as disclosing the limitations of 
Appellants' claim 1. Appellants note, for example, Escobar at col. 4 lines 33-39 describes 
executing "the IDL" at either an end-user location or a server location to control "playback of 
running of the interactive multimedia application." Appellants submit that this description 
merely implies that the multimedia playback assets/objects may be retrieved from storage and 
assembled in a suitable time sequence at either user or server locations. The cited portions of 
Escobar neither mention nor support an inference of streaming scene composition information 
for objects, as recited in the claimed subject matter. 

For all of the above reasons, Appellants respectfully submit that the rejections of 

record of claim 1 as anticipated by Escobar should be reversed. The rejections of corresponding 

depending claims 2-7 should also be reversed for at least the same reasons discussed above. 

2. Claims 8-10 Are Not Anticipated Because Escobar 
Does Not Disclose "controlling acquisition over time 
of streaming data including a plurality of audio- 
visual/video objects and composition information 
for the objects" 

Independent claims 8, 9 and 10 each contain limitations similar to those discussed 
above with respect to claim 1 . For at least those reasons discussed above with respect to claim 1, 
because Escobar fails to disclose or suggest the claimed "controlling acquisition over time of 
streaming data including a plurality of audio-visual/video objects and composition information 
for the objects," the rejections of claims 8-10 should also be reversed. 
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3. Claims 1-7 Are Also Not Anticipated Because Escobar 
Does Not Disclose "processing the composition 
information received in the data bitstream to compose 
scenes from the received objects" 

Additionally, Escobar does not disclose or suggest "processing the composition 
information received in the data bitstream to compose scenes" as recited in Appellants' claim 1. 

Escobar at col. 4, lines 16-18 and col. 12, lines 46-47, merely describes arranging 
or positioning the multimedia assets/objects in a time sequence for playback. As discussed 
above, the assets may include "video assets from a video recorder, a variety of sounds recorded 
by an audio engineer, the film output from movie cameras with or without related audio tracks, 
and perhaps different films from different camera locations of the same scene shot by different 
cameras." (Escobar, col. 7 lines 35-48). The assets or objects of Escobar are just digitized 
versions of these items, such as video clips. Id. The already-completed video clips of Escobar 
are not "composed," and, indeed, no composition information is required to display the video 
clips of Escobar. As a result, the cited portions of Escobar neither mention nor support inference 
of "processing the composition information" to compose the objects in a scene. 

Thus, contrary to the allegations in the Office Action, Escobar does not disclose 
or suggest " processing the composition information received in the data bitstream to compose 
scenes from the received objects," as recited in Appellants' claim 1 . Additionally, this claim 
limitation is included in claims 2-7 via dependency from claim 1 . Accordingly, for at least these 
additional reasons, Appellants respectfully submit that Escobar fails to disclose or suggest all 
elements of claims 1-7 and, therefore, cannot anticipate Appellants' claimed subject matter. 
Appellants respectfully request reversal of all rejections of record for at least these additional 
reasons. 
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4. Claims 8-10 Are Also Not Anticipated Because Escobar 
Does Not Disclose "processing the composition 
information acquired in the streaming data to compose 
scenes from said video objects including the one of the 
objects stored in the cache memory" 

Independent claims 8, 9 and 10 each contain limitations similar to those discussed 
above with respect to claim 1 . For at least those reasons discussed above with respect to claim 1 , 
because Escobar fails to disclose or suggest the claimed "processing the composition information 
acquired in the streaming data to compose scenes from said video objects including the one of 
the objects stored in the cache memory," the rejections of claims 8-10 should also be reversed. 
B. Conclusion 

For at least the reasons discussed above, Appellants respectfully submit that the 
invention recited in the claims of the present application, as discussed above, is not anticipated 
by the cited prior art. Reversal of the Examiner's rejections of the claims is therefore 
respectfully requested. 

Respectfully submitted, 



Dated: September 8, 2006 By: 




Paul A. Ragusa 

Patent Office Reg. No. 38,587 



Robert L. Maier 

Patent Office Reg. No. 54,291 

Attorneys for Appellants 
Baker Botts L.L.P. 
30 Rockefeller Plaza 
New York, NY 10112-4498 
Telephone: (212) 408-2500 
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VIII. CLAIMS APPENDIX 

Claims 1-10 are pending in this application; 

1 . A method for displaying object-based audiovisual/video data, comprising: 

(a) streaming in a data bit stream, over time, a plurality of 
audiovisual/video objects and composition information for the objects, to a receiver; 

(b) at the receiver, storing in a cache memory at least one of the objects; 

(c) processing the composition information received in the data bitstream 
to compose scenes from said objects including the one of the objects stored in the cache 
memory; and 

(d) displaying the composed scenes. 

2. The method of claim 1 , further comprising, in addition to storing the one 
of the objects, storing expiration time data for the one of the objects. 

3. The method of claim 1, with at least one of the objects being received 
from a network connection. 

4. The method of claim 1 , with at least one of the objects being received 
from local memory. 

5. The method of claim 1, with at least one of the objects being received 
from local memory and at least one other of the objects being received from a network 
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connection, and with the composed scenes comprising the one and the other of the 
objects. 

6. The method of claim 1 , further comprising responding to interactive user 

input. 

7. The method of claim 6, wherein responding comprises at least one of 
selecting, enabling and disenabling one of the objects. 

8. Apparatus for displaying object-based audiovisual/video data, comprising, 

(a) a controller circuit for controlling acquisition over time of streaming 
data including a plurality of audio visual/video objects and composition information for 
the objects; 

(b) a cache memory for storing at least one of the objects; 

(c) a composer circuit, coupled to the cache memory, for processing the 
composition information acquired in the streaming data to compose scenes from said 
video objects including the one of the objects stored in the cache memory; and 

(d) a display for the composed scenes. 

9. Apparatus for displaying object-based audiovisual/video data, comprising 
a processor which is instructed for: 

(a) controlling acquisition over time of streaming data including a plurality 
of audio-visual/video objects and composition information for the objects; 
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(b) storing in a cache memory at least one of the objects; 

(c) processing the composition information acquired in the streaming data 
to compose scenes from said video objects including the one of the objects stored in the 
cache memory; and 

(d) displaying the composed scenes. 

10. Apparatus for displaying object-based audiovisual/video data, comprising: 

(a) means for controlling acquisition over time of streaming data including 
a plurality of audio-visual/video objects and composition information for the objects; 

(b) means for storing in a cache memory at least one of the objects; 

(c) means, coupled to the cache memory, for processing the composition 
information acquired in the streaming data to compose scenes from said video objects 
including the one of the objects stored in the cache memory; and 

(d) means for displaying the scenes. 
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IX. EVIDENCE APPENDIX 

None. 
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X. RELATED PROCEEDINGS APPENDIX 

None. 
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